Skip to content

Improve reusable deploy workflow diagnostics for webhook failures and non-2xx responses - #16

Merged
GeneralTao2 merged 14 commits into
mainfrom
copilot/update-deployment-workflow
May 12, 2026
Merged

Improve reusable deploy workflow diagnostics for webhook failures and non-2xx responses#16
GeneralTao2 merged 14 commits into
mainfrom
copilot/update-deployment-workflow

Conversation

Copilot AI commented May 11, 2026

Copy link
Copy Markdown
Contributor
  • Reproduce the failing integration test locally and inspect webhook/runtime logs
  • Adjust deploy error detection so valid deployments still succeed while real failures still fail, preserving optional health-check behavior
  • Update focused regression coverage and any affected docs
  • Re-run the relevant test entrypoints and validation
Original prompt

Update the reusable deployment workflow in taonity/docker-webhook so GitHub Actions gets clearer deployment results and better failure diagnostics.

Context:

  • The workflow file is .github/workflows/deploy.yml.
  • It is used by downstream repositories like taonity/artist-insight-service to trigger deployments by POSTing to the docker-webhook service.
  • The current implementation uses a single curl --no-buffer --fail-with-body --max-time 300 ... call and prints a success message afterward.
  • A failing downstream job showed curl: (22) The requested URL returned error: 504, which makes it hard to distinguish proxy/webhook failures from deployment failures and does not reliably surface the HTTP status/body in a structured way.

Requested changes:

  1. Modify .github/workflows/deploy.yml in taonity/docker-webhook.
  2. Preserve the existing contract and inputs (project_name, environment, WEBHOOK_URL, WEBHOOK_SECRET).
  3. Improve the shell step so it:
    • captures the HTTP response body to a file,
    • captures and prints the HTTP status code,
    • prints the response body to the GitHub Actions log,
    • uses a longer timeout (e.g. 600 seconds) instead of 300,
    • fails explicitly on non-2xx responses,
    • keeps the output focused and readable.
  4. Ensure the workflow still exits non-zero when deployment fails.
  5. Keep the change minimal and consistent with the existing YAML style.

Deliverable:

  • Create a pull request with the workflow update in taonity/docker-webhook based on the main branch.

The following is the prior conversation context from the user's chat exploration (may be truncated):

User: Please find a solution for this failing job. Use the logs, job definition, and any referenced files where the failure occurred. Keep your response focused on the solution and include code suggestions when appropriate.

Assistant: The failure is in the deploy job’s webhook call, not in the build or image-push steps.

Root cause

The job failed on the curl POST used by the reusable deploy workflow:

  • Workflow: release.yml
  • Trigger point: deploy-stage at lines 163–171, which calls taonity/docker-webhook/.github/workflows/deploy.yml@main
  • Log failure:
    • curl: (22) The requested URL returned error: 504
    • exit code 22

A 504 means the webhook endpoint behind WEBHOOK_URL did not complete in time. This is an infrastructure/deployment endpoint timeout, not a GitHub Actions syntax issue.

What to fix

Update the reusable deploy workflow so the webhook call is more resilient and easier to diagnose.

Recommended changes

  1. Retry transient 5xx failures
  2. Reduce hard failure from temporary gateway timeouts
  3. Log response body and HTTP status clearly
  4. Use shorter connect timeout and controlled retries instead of one long request

Suggested code change

In the reusable deploy workflow taonity/docker-webhook/.github/workflows/deploy.yml@main, replace the current curl with something like:

echo "Deploying project [$PROJECT_NAME] to [$ENVIRONMENT]"
echo "---"

http_code=$(
  curl --no-buffer --silent --show-error \
    --output response.txt \
    --write-out "%{http_code}" \
    --retry 5 \
    --retry-all-errors \
    --retry-delay 10 \
    --connect-timeout 10 \
    --max-time 60 \
    -X POST \
    -H "Content-Type: application/json" \
    -H "X-Webhook-Secret: $WEBHOOK_SECRET" \
    -d "{\"project_name\":\"$PROJECT_NAME\",\"environment\":\"$ENVIRONMENT\"}" \
    "$WEBHOOK_URL"
)

echo "HTTP status: $http_code"
cat response.txt

if [ "$http_code" -lt 200 ] || [ "$http_code" -ge 300 ]; then
  echo "Deployment webhook failed with status $http_code"
  exit 1
fi

Why this helps

The current request appears to wait up to 300 seconds and still ends with a gateway timeout. That usually means:

  • the upstream deployment service is too slow,
  • a proxy/load balancer times out before the backend finishes,
  • or the webhook handler is doing synchronous work that should be asynchronous.

The change above makes the workflow tolerate temporary 504 responses and gives clearer output.

Stronger fix on the webhook service

If you control the webhook server, the better solution is:

  • make the webhook return quickly with 202 Accepted,
  • queue the deployment asynchronously,
  • let the actual deployment continue in the background.

That avoids GitHub Actions waiting on a long-running HTTP request that may hit proxy time limits.

Repo-level workflow context

The relevant job definition in this repo is:

deploy-stage:
  needs: release
  uses: taonity/docker-webhook/.github/workflows/deploy.yml@main
  with:
    project_name: artist-insight...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

*This pull request was created from Copilot chat.*
>

Copilot AI changed the title [WIP] Update deployment workflow for clearer results and diagnostics Improve reusable deploy workflow diagnostics for webhook failures and non-2xx responses May 11, 2026
Copilot AI requested a review from GeneralTao2 May 11, 2026 21:01
Copilot AI and others added 8 commits May 11, 2026 21:21
Agent-Logs-Url: https://github.com/taonity/docker-webhook/sessions/8c98fc1f-d103-4c8c-808b-3beaa6f0c7f4

Co-authored-by: GeneralTao2 <42372666+GeneralTao2@users.noreply.github.com>
Agent-Logs-Url: https://github.com/taonity/docker-webhook/sessions/8c98fc1f-d103-4c8c-808b-3beaa6f0c7f4

Co-authored-by: GeneralTao2 <42372666+GeneralTao2@users.noreply.github.com>
Agent-Logs-Url: https://github.com/taonity/docker-webhook/sessions/8c98fc1f-d103-4c8c-808b-3beaa6f0c7f4

Co-authored-by: GeneralTao2 <42372666+GeneralTao2@users.noreply.github.com>
Agent-Logs-Url: https://github.com/taonity/docker-webhook/sessions/8c98fc1f-d103-4c8c-808b-3beaa6f0c7f4

Co-authored-by: GeneralTao2 <42372666+GeneralTao2@users.noreply.github.com>
Agent-Logs-Url: https://github.com/taonity/docker-webhook/sessions/8c98fc1f-d103-4c8c-808b-3beaa6f0c7f4

Co-authored-by: GeneralTao2 <42372666+GeneralTao2@users.noreply.github.com>
Agent-Logs-Url: https://github.com/taonity/docker-webhook/sessions/8c98fc1f-d103-4c8c-808b-3beaa6f0c7f4

Co-authored-by: GeneralTao2 <42372666+GeneralTao2@users.noreply.github.com>
Agent-Logs-Url: https://github.com/taonity/docker-webhook/sessions/8c98fc1f-d103-4c8c-808b-3beaa6f0c7f4

Co-authored-by: GeneralTao2 <42372666+GeneralTao2@users.noreply.github.com>
Agent-Logs-Url: https://github.com/taonity/docker-webhook/sessions/8c98fc1f-d103-4c8c-808b-3beaa6f0c7f4

Co-authored-by: GeneralTao2 <42372666+GeneralTao2@users.noreply.github.com>
Copilot AI and others added 3 commits May 12, 2026 10:01
Agent-Logs-Url: https://github.com/taonity/docker-webhook/sessions/f8db322e-3d51-4f77-ad06-6994084e5b64

Co-authored-by: GeneralTao2 <42372666+GeneralTao2@users.noreply.github.com>
Agent-Logs-Url: https://github.com/taonity/docker-webhook/sessions/f8db322e-3d51-4f77-ad06-6994084e5b64

Co-authored-by: GeneralTao2 <42372666+GeneralTao2@users.noreply.github.com>
Agent-Logs-Url: https://github.com/taonity/docker-webhook/sessions/f8db322e-3d51-4f77-ad06-6994084e5b64

Co-authored-by: GeneralTao2 <42372666+GeneralTao2@users.noreply.github.com>
@GeneralTao2
GeneralTao2 marked this pull request as ready for review May 12, 2026 10:19
@GeneralTao2
GeneralTao2 merged commit e7e337c into main May 12, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants